Image processing apparatus for converting image in characteristic region of original image into image of brushstroke patterns

ABSTRACT

An object of the present invention is to obtain an image that is more similar to a real ink-wash painting. An ink-wash painting conversion unit  44  converts data of an original image into data of a painterly image. A characteristic region detection unit  42  detects a characteristic region of the original image from the data of the original image. A conversion unit  45  executes gradation processing of gradating the characteristic region detected by the characteristic region detection unit  42,  and margin setting processing of setting a margin region to be added to the painterly image, as image processing of further converting the data of the painterly image that was converted by the ink-wash painting conversion unit  44.

This application is based on and claims the benefit of priority fromJapanese Patent Application No. 2011-213375, respectively filed on 28Sep. 2011, the content of which is incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing apparatus and animage processing method as well as a storage medium.

2. Related Art

In recent years, image processing is executed on data of an originalimage to improve artistry for the purpose of enhancing the renderingeffects.

For example, Japanese Unexamined Patent Application, Publication No.2011-139329 discloses painterly conversion processing, which is executedon data of an original image to be converted into data of a painterlyimage such as an image similar to an oil painting or a colored pencildrawing.

Moreover, in order to accomplish the aforementioned purpose, JapaneseUnexamined Patent Application, Publication No. 2006-114024 disclosesimage processing, in which an edge is detected in data of an originalimage that includes a person's face as a subject, thereby converting thedata of the original image into data of an image that is similar to anink-wash painting (suiboku-ga) (hereinafter referred to as an“ink-wash-painting-like image”) that is a type of an image with highartistry.

However, in the image processing disclosed in Japanese Unexamined PatentApplication, Publication No. 2006-114024, only a width of a contour lineis converted depending on a facial part, and for example,ink-wash-painting expression is not taken into consideration at all. Asa result, a converted image may be an image being remote from a realink-wash painting in some cases.

SUMMARY OF THE INVENTION

The image processing apparatus according to one aspect of the presentinvention is characterized by including:

a first conversion unit that converts data of an original image intodata of a painterly image; a characteristic region detection unit thatdetects a characteristic region of the original image, from the data ofthe original image; and

a second conversion unit that executes gradation processing of gradatingthe characteristic region detected by the characteristic regiondetection unit, and margin setting processing of setting a margin regionto be added to the painterly image, as image processing of furtherconverting the data of the painterly image that was converted by thefirst conversion unit.

In addition, an image processing method performed by an image processingapparatus to execute image processing on an original image according toone aspect of the present invention, the method including:

a first converting step of converting data of the original image intodata of a painterly image;

a characteristic region detecting step of detecting a characteristicregion of the original image, from the data of the original image; and

a second converting step of executing gradation processing of gradatingthe characteristic region detected in the characteristic regiondetecting step, and margin setting processing of setting a margin regionto be added to the painterly image, as image processing of furtherconverting the data of the painterly image that was converted by thefirst converting step.

In addition, a storage medium having stored therein a computer readableprogram for controlling an image processing apparatus that executesimage processing on an original image according to one aspect of thepresent invention, the program causing a computer to implement functionsof:

a first converting unit that converts data of the original image intodata of a painterly image;

a characteristic region detection unit that detects a characteristicregion of the original image, from the data of the original image; and

a second conversion unit that executes gradation processing of gradatingthe characteristic region detected by the characteristic regiondetection unit, and margin setting processing of setting a margin regionto be added to the painterly image, as image processing of furtherconverting the data of the painterly image that was converted by thefirst conversion unit.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a hardware configuration of an imageprocessing apparatus according to the first embodiment of the presentinvention;

FIG. 2 is a functional block diagram showing a functional configurationfor executing ink-wash-painting-like image generation processing,overall gradation processing, and margin region setting processing, in afunctional configuration of the image processing apparatus shown in FIG.1;

FIG. 3 shows an example of data of an original image;

FIG. 4A is a diagram showing an example of a brushstroke pattern;

FIG. 4B is a diagram showing an example of a brushstroke pattern;

FIG. 5 is a diagram showing examples of edge images;

FIG. 6 a diagram showing examples of ink-wash-painting-like images thatare converted from the edge images shown in FIG. 5;

FIG. 7A shows an example of an image after setting a margin region to beadded to an ink-wash-painting-like image, by way of anink-wash-painting-like image and a white background image;

FIG. 7B shows an example of an image after setting a margin region to beadded to an ink-wash-painting-like image, by way of anink-wash-painting-like image and a white background image;

FIG. 7C shows an example of an image after setting a margin region to beadded to an ink-wash-painting-like image, by way of anink-wash-painting-like image and a white background image;

FIG. 8 is a flowchart showing an example of a flow ofink-wash-painting-like image processing that is executed by the imageprocessing apparatus shown in FIG. 1 having the functional configurationshown in FIG. 2;

FIG. 9 is a flowchart illustrating importance region calculationprocessing, regarding the ink-wash-painting-like image processing shownin FIG. 8;

FIG. 10 is a flowchart illustrating ink-wash-painting-like imagegeneration processing, regarding the ink-wash-painting-like imageprocessing shown in FIG. 8;

FIG. 11 is a flowchart illustrating overall gradation processing,regarding the ink-wash-painting-like image processing shown in FIG. 8;

FIG. 12 is a flowchart illustrating margin region setting processing,regarding the ink-wash-painting-like image processing shown in FIG. 8;

FIG. 13 is a functional block diagram showing a functional configurationfor executing overall gradation processing according to a secondembodiment;

FIG. 14 shows an example of data of an original image of the secondembodiment; and

FIG. 15 is a flowchart illustrating the overall gradation processing,regarding the ink-wash-painting-like image processing of the secondembodiment.

DETAILED DESCRIPTION OF THE INVENTION First Embodiment

In the following, a first embodiment of the present invention will beexplained with reference to the drawings.

FIG. 1 is a block diagram showing a hardware configuration of an imageprocessing apparatus according to the first embodiment of the presentinvention.

The image processing apparatus 1 is provided with a CPU (CentralProcessing Unit) 11, ROM (Read Only Memory) 12, RAM (Random AccessMemory) 13, an image processing unit 14, a bus 15, an input/outputinterface 16, an input unit 17, an output unit 18, a storage unit 19, acommunication unit 20, and a drive 21.

The CPU 11 executes various processing according to programs that arerecorded in the ROM 12, or programs that are loaded from the storageunit 19 to the RAM 13.

The RAM 13 also stores data and the like necessary for the CPU 11 toexecute the various processing, as appropriate.

The image processing unit 14 is configured by a DSP (Digital SignalProcessor), VRAM (Video Random Access Memory) and the like, andcollaborates with the CPU 11 to execute various image processing onimage data. Although not described in detail, the image processing unit14 also includes a function of executing processing on data of anoriginal image to be converted into data of a painterly image such as animage similar to an oil painting or a colored pencil drawing.

The CPU 11, the ROM 12, the RAM 13, and the image processing unit 14 areconnected to one another via the bus 15. The bus 15 is also connectedwith the input/output interface 16. The input/output interface 16 isconnected to the input unit 17, the output unit 18, the storage unit 19,the communication unit 20, and the drive 21.

The input unit 17 is configured by a keyboard, a mouse and the like, andinputs various information in accordance with instruction operations bya user.

The output unit 18 is configured by a display, a speaker and the like,and outputs images and sound.

The storage unit 49 is configured by DRAM (Dynamic Random Access Memory)or the like, and stores data of various images.

The communication unit 20 controls communication with other devices (notshown) via a network, which includes the Internet.

Removable media 31 made of a magnetic disk, an optical disk, amagneto-optical disk, semiconductor memory or the like is installed inthe drive 21, as appropriate. Programs that are read via the drive 21from the removable media 31 are installed in the storage unit 19, asnecessary. In addition, similarly to the storage unit 19, the removablemedia 31 can also store various data such as the image data stored inthe storage unit 19.

FIG. 2 is a functional block diagram showing a functional configurationfor executing ink-wash-painting-like image generation processing,overall gradation processing, and margin region setting processing, inthe functional configuration of such an image processing apparatus 1.

Here, the ink-wash-painting-like image generation processing refers to asequence of processing of generating data of an ink-wash-painting-likeimage from data of an original image that is input as an object of imageprocessing.

The overall gradation processing refers to a sequence of processing ofgradating a characteristic region in the ink-wash-painting-like image,based on importance information calculated from the data of the originalimage.

Moreover, the margin region setting processing refers to a sequence ofprocessing of setting a margin region to be added to theink-wash-painting-like image, based on the importance information.

In a case in which the ink-wash-painting-like image generationprocessing is executed according to control by the CPU 11, an originalimage acquisition unit 41, a characteristic region detection unit 42,and an ink-wash painting conversion unit 44 function in the imageprocessing unit 14. In a case in which the overall gradation processingis executed according to control by the CPU 11, the original imageacquisition unit 41, an importance information calculation unit 43, anda conversion unit 45 function in the image processing unit 14.

Moreover, in a case in which the margin region setting processing isexecuted according to control by the CPU 11, the importance informationcalculation unit 43 and the conversion unit 45 function in the imageprocessing unit 14.

In addition, a brushstroke pattern storage unit (not shown) is providedas an area of the storage unit 19.

The original image acquisition unit 41 acquires data of an imagetransmitted from another device and received by the communication unit20, data of an image that is read from the removable media 31 via thedrive 21 and the like as data of the original image, and supplies suchdata to the characteristic region detection unit 42 and the importanceinformation calculation unit 43.

The characteristic region detection unit 42 executes edge detectionprocessing on data of the original image, and generates data of a binaryimage, in which, for example, the detected edge is black, and the otherportions are white (such a binary image is hereinafter referred to as an“edge image”). The characteristic region detection unit 42 detects acharacteristic region for data of the edge image thus generated.Although the object detected as a characteristic region is not limitedin particular, it is an edge region in the present embodiment. The dataof the edge image thus generated by the characteristic region detectionunit 42 is supplied to the ink-wash painting conversion unit 44.

The importance information calculation unit 43 detects importance ofeach pixel composing an original image, by using a color, brightness, anedge direction or the like as an indicator of importance, from the dataof the original image acquired by the original image acquisition unit41. In addition, the importance information calculation unit 43generates an importance map indicating spatial distribution of theimportance detected for each pixel, thereby executing importance regioncalculation processing of calculating importance information of theoriginal image.

More specifically, for example, the importance information calculationunit 43 calculates importance information by executing the importanceregion calculation processing on data of the original image as shown inFIG. 3. The importance information thus calculated by the importanceinformation calculation unit 43 is supplied to the ink-wash paintingconversion unit 44 and the conversion unit 45.

FIG. 3 shows an example of data of an original image 100.

In FIG. 3, the importance region 102 is a gray region, among each of aplurality of pixels 101 composing the original image 100. In addition,an importance barycenter 111 calculated based on the importance region102 is shown in a substantially central portion of the importance region102.

More specifically, as shown in FIG. 2, in order to execute theimportance region calculation processing as such, the importanceinformation calculation unit 43 of the present embodiment includes animportance calculation unit 51, an importance region calculation unit52, and an importance barycenter calculation unit 53.

The importance calculation unit 51 calculates importance of each pixelcomposing the original image acquired by the original image acquisitionunit 41. For example, in the example shown in FIG. 3, importance of eachpixel composing the original image 100 is calculated. The calculation ofimportance is not limited in particular, but in the present embodiment,the importance calculation unit 51 calculates importance, based on colorinformation, brightness (brightness information), or information of anedge direction, within a predetermined range around each pixel. In thiscase, since importance cannot be calculated based on only a singlepixel, the importance calculation unit 51 calculates importance based onwhether pixels as representative values are important for a peripheralregion around a particular region with respect to an attention pixel.More specifically, when detecting an edge in a direction different fromthe direction of the edge around the attention pixel, the importancecalculation unit 51 recognizes such a portion as an object having acolor different from its peripheral colors, thereby recognizing that acharacteristic object is drawn in the portion, and increasing theimportance of the portion. The importance information calculated by theimportance calculation unit 51 is supplied to the importance regioncalculation unit 52 and the importance barycenter calculation unit 53.

Based on the importance calculated for each pixel by the importancecalculation unit 51, the importance region calculation unit 52calculates an importance region, for example, the importance region 102in the example shown in FIG. 3. The calculation of the importance regionis not limited in particular, but in the present embodiment, in a casein which an upper limit of an importance evaluation value calculated bythe importance calculation unit 51 is assumed to be 100, a region withan importance evaluation value being 90 or higher can be calculated asan importance region. The information of the importance regioncalculated by the importance region calculation unit 52 is supplied tothe ink-wash painting conversion unit 44 and the conversion unit 45.

Based on the importance of each pixel calculated by the importancecalculation unit 51, the importance barycenter calculation unit 53calculates an importance barycenter of the original image. Thecalculation of an importance barycenter is not limited in particular,but in the present embodiment, when the importance region calculationunit 52 calculates an importance region, the importance barycentercalculation unit 53 calculates a central position in such a region as animportance barycenter. More specifically, at first, the importancebarycenter calculation unit 53 calculates an importance barycenter (x,y) in spatial distribution of the importance. The importance barycentercalculation unit 53 calculates the importance barycenter (x, y) byapplying Equations (1) and (2) as follows to pixels having importance ofat least a threshold value. The threshold value can be appropriatelychanged by the user as necessary, and in the present embodiment, thethreshold value can be 90 in a case in which the upper limit of theimportance is 100.

mx=1/(n+sumw)*Σ(x*w(x, y))   (1)

mx=1/(n+sumw)*Σ(y*w(x, y))   (2)

(where n represents the number of pixels having importance of at least athreshold value; sumw represents a sum of the importance; and w (x, y)represents importance in (x, y)).

The importance barycenter information calculated by the importanceinformation calculation unit 43 is supplied to the conversion unit 45.

The ink-wash painting conversion unit 44 executes processing ofconverting the data of the edge image into data of anink-wash-painting-like image. Such processing is hereinafter referred toas “ink-wash painting conversion processing”.

As the ink-wash painting conversion processing employed in the presentembodiment, a texture that imitates handwriting of a writing brush forcalligraphy is applied to an original image. Such a pattern of the“texture that imitates handwriting of a writing brush for calligraphy”is referred to as a “brushstroke pattern” in the present specification.

A shape, a size and the like of the texture employed as the brushstrokepattern are not limited in particular. However, two types of brushstrokepatterns shown in FIG. 4 are employed in the present embodiment.

FIG. 4 is a diagram showing examples of the brushstroke patterns.

FIG. 4A shows a brushstroke pattern that imitates handwriting in a caseof using a portion “the longest hair on the tip” of the writing brush(an extreme tip portion of the writing brush). It should be noted thatthe brushstroke pattern as shown in FIG. 4A is hereinafter referred toas an “upright-brush brushstroke pattern”.

FIG. 4B shows a brushstroke pattern that imitates handwriting in a caseof using a lateral portion (a part of the “head”) of the writing brushranging from a “neck (portion adjacent to the extreme tip)” to a“ventral (middle portion)” thereof. It should be noted that thebrushstroke pattern as shown in FIG. 4B is hereinafter referred to as an“oblique-brush brushstroke pattern”.

As would be easily understood by comparing FIGS. 4A and 4B, theupright-brush brushstroke pattern is characterized in that a length in adirection orthogonal to a longitudinal direction (hereinafter referredto as a “width”) is narrow, as a result of which the variation in thegray levels is small. In contrast, the oblique-brush brushstroke patternis characterized in that the width is wide, as a result of which thevariation in the gray levels is large.

In the present embodiment, each data of the upright-brush brushstrokepattern and the oblique-brush brushstroke pattern as described above isstored in a brushstroke pattern storage unit (not shown).

It should be noted that a technique for generating data of thebrushstroke patterns is not limited in particular, and for example, atechnique for generating data by using CG (Computer Graphics) may beemployed. However, the present embodiment employs a technique, in whichhandwriting actually written on a paper medium with a writing brushsoaked in ink (Japanese Sumi) is captured with a scanner or a digitalcamera to create data of brushstroke patterns.

The ink-wash painting conversion unit 44 executes ink-wash paintingconversion processing on data of an edge image, in which theupright-brush brushstroke pattern is applied to an edge region, theoblique-brush brushstroke pattern is applied to a surrounding region ofthe edge region, and in a case in which the edge region forms a closedregion, the oblique-brush brushstroke pattern is applied to an inside ofthe closed region.

More specifically, for example, the ink-wash painting conversion unit 44executes the ink-wash painting conversion processing on data of an edgeimage as shown in FIG. 5, thereby generating data of anink-wash-painting-like image as shown in FIG. 6.

FIG. 5 shows examples of edge images.

In FIG. 5, a black and narrow area is the edge region.

For example, an edge region E1 shows a contour of a mountain, an edgeregion E2 shows a contour of a cloud, and an edge region E3 shows acontour of a house.

FIG. 6 shows examples of ink-wash-painting-like images that areconverted from the edge images in FIG. 5.

An upright-brush brushstroke pattern D1 is applied to the edge region E1showing the contour of the mountain, and an oblique-brush brushstrokepattern S1 is applied to a surrounding region on the right side of theedge region E1.

Moreover, since the edge region E2 showing the contour of the cloudforms a closed region, an upright-brush brushstroke pattern D2 isapplied to the edge region E2, and an oblique-brush brushstroke patternS2 is applied so as to paint out an inside of the closed region.

Similarly, since the edge region E3 showing the contour of the houseforms a closed region, an upright-brush brushstroke pattern D3 isapplied to the edge region E3, and an oblique-brush brushstroke patternS3 is applied so as to paint out an inside of the closed region.

More specifically, in order to execute the ink-wash painting conversionprocessing as described above, the ink-wash painting conversion unit 44of the present embodiment executes processing as follows.

More specifically, the ink-wash painting conversion unit 44 detects acharacteristic region for data of the edge image. Although the objectdetected as a characteristic region is not limited in particular, it isan edge region in the present embodiment.

Based on the characteristic region thus detected, the ink-wash paintingconversion unit 44 determines a brushstroke pattern to be used fromamong brushstroke patterns, of which data is stored in the brushstrokepattern storage unit (not shown).

More specifically, in the present embodiment, the ink-wash paintingconversion unit 44 determines an upright-brush brushstroke pattern asthe brushstroke pattern to be used for the edge region that is thecharacteristic region. Moreover, the ink-wash painting conversion unit44 determines an oblique-brush brushstroke pattern as the brushstrokepattern to be used for the surrounding region of the edge region that isthe characteristic region, or for the region inside a closed curve ofthe edge region that forms the closed curve.

The ink-wash painting conversion unit 44 converts the data of the edgeimage into data of an ink-wash-painting-like image by using data of thebrushstroke pattern thus determined.

More specifically, the ink-wash painting conversion unit 44 converts thedata of the edge region into data of the upright-brush brushstrokepattern, such that the upright-brush brushstroke pattern is applied tothe edge region. Similarly, the ink-wash painting conversion unit 44converts the data of the surrounding region of the edge region into dataof the oblique-brush brushstroke pattern, such that the oblique-brushbrushstroke pattern is applied to the surrounding region of the edgeregion. In addition, in a case in which the edge region forms a closedregion, the ink-wash painting conversion unit 44 converts data of theinside of the closed region into data of the oblique-brush brushstrokepattern, such that the oblique-brush brushstroke pattern is applied soas to paint out the inside of the closed region.

Based on the importance map generated by the importance informationcalculation unit 43, the ink-wash painting conversion unit 44 adjustseach color of the brushstroke pattern used in the ink-wash paintingconversion unit 44, i.e. adjusts the gray levels of the ink.

More specifically, as described above, the importance map shows thespatial distribution of the importance of each pixel. On the other hand,the gray level is determined for each brushstroke pattern occupying aregion consisting of a plurality of pixel groups.

Accordingly, the ink-wash painting conversion unit 44 extracts, from theimportance map, each importance of the plurality of pixel groupsincluded in the region occupying the brushstroke pattern of anadjustment object, and calculates integrated importance of the entireregion, based on a plurality of pieces of importance thus extracted.

It should be noted that a technique for calculating integratedimportance of the entire region is not limited in particular as long asimportance of a plurality of pixels composing the region is used, andfor example, a technique for calculating a root mean square and a meanvalue can be employed. However, in order to easily calculate importancewith a small number of calculations in short time, a technique forcalculating a simple average of importance of a plurality of pixels isemployed in the present embodiment.

The ink-wash painting conversion unit 44 adjusts the gray levels, suchthat the brushstroke pattern occupying the region is darker (verges onblack) as the integrated importance of the entire region is increased,whereas the brushstroke pattern occupying the region is lighter (vergeson white) as the integrated importance of the entire region isdecreased.

In order to express a blur of the ink, the ink-wash painting conversionunit 44 executes image processing of gradating from the region of theprocessing object to its perimeter, for each data of the region that wasconverted into (applied as) a brushstroke pattern by the ink-washpainting conversion unit 44 (this image processing is hereinafterreferred to as “gradation processing”).

The amount (width) of gradation in the gradation processing isdetermined depending on the gray levels in the region of the processingobject. More specifically, since the gray levels in the region of theprocessing object are adjusted based on the importance map as describedabove, the amount of gradation in the region of the processing object isalso determined based on the importance map. In this case, as theimportance is higher, the color is darker (verges on black), and theamount of gradation is smaller; conversely, as the importance is lower,the color is lighter (verges on white), and the amount of gradation isgreater.

Moreover, the manner of gradating each pixel may be a technique thatdepends on a distance x from an edge of a region of a processing object(a brushstroke pattern), and is not limited in particular; however, thepresent embodiment employs a technique, in which the color is lighter asthe distance x is increased. More specifically, the present embodimentemploys a technique, in which the gray scales of an image (a range ofbrightness indicating the gray levels) are 256 gray scales, and thecolor gradation (brightness indicating the gray levels) of a pixel of aprocessing object is calculated according to Equation (3) as follows.

B=(255−L)*(1−exp(−x*x/f(D+n)))+L   (3)

In Equation (3), B represents the color gradation (brightness indicatingthe gray levels) of a pixel of a processing object. L represents a colorof the brushstroke pattern applied to the pixel of the processing object(brightness indicating the gray levels regarding the entire region ofthe processing object). f(D+n) represents an arbitrary function, ofwhich output value is increased in accordance with an input parameter(D+n). D represents an amount of gradation in the brushstroke pattern(the region of the processing object) applied to the pixel of theprocessing object. n represents an arbitrary integer.

The conversion unit 45 includes: a gradation unit 61 that executesoverall gradation processing on data of the ink-wash-painting-like imagein the characteristic region, based on the importance informationcalculated by the importance information calculation unit 43; and amargin region setting unit 62 that executes margin region settingprocessing of setting a margin region of the original image.

Based on the importance of each pixel in the importance regioncalculated by the importance region calculation unit 52, and based on adistance between each pixel and the importance barycenter calculated bythe importance barycenter calculation unit 53, the gradation unit 61calculates an amount of gradation in each pixel. More specifically, thegradation unit 61 calculates an amount of gradation such that the amountof gradation in the edge is maximized, by gradually changing thebrightness toward the white gray scale (256), based on primaryapproximation by a predetermined function and the Gaussian distribution(normal distribution). In other words, the gradation processing dependson a distance x from the edge of the brush, and the color is lighter asthe distance from the edge of the brush is increased. Here, when theamount of gradation is D, and the color (brightness) of the brushdrawing is L, a gradation color B is expressed by Equation (4) asfollows.

B=(255−L)*(1−exp(−x*x/f(D+n)))+L   (4)

In the present embodiment, 256 gray scales are used as the gray scalesof an image, and a function f represents an arbitrary function that isincreased according to a value of D+n. Moreover, n represents anarbitrary integer.

The gradation unit 61 executes gradation processing on the entire imageof the ink-wash-painting-like image, based on the amount of gradationcalculated for each pixel.

The margin region setting unit 62 executes margin region settingprocessing on data of the ink-wash-painting-like image, thereby settinga margin region to be added to the ink-wash-painting-like image.

More specifically, as shown in FIG. 2, in order to execute the marginregion setting processing as such, the margin region setting unit 62 ofthe present embodiment includes an importance barycenter position ratiocalculation unit 71, a background image position ratio calculation unit72, and a synthesis unit 73.

The importance barycenter position ratio calculation unit 71 calculatesa position ratio of the importance barycenter calculated by theimportance barycenter calculation unit 53.

Based on the position ratio of the importance barycenter calculated bythe importance barycenter position ratio calculation unit 71, thebackground image position ratio calculation unit 72 calculates abarycenter position ratio of a white background image.

Based on the position ratio of the importance barycenter calculated bythe importance barycenter position ratio calculation unit 71, and basedon the position ratio of the barycenter of the background imagecalculated by the background image position ratio calculation unit 72,the synthesis unit 73 executes synthesis processing such that the dataof the ink-wash-painting-like image, on which the gradation processingwas executed by the gradation unit 61, is synthesized so as to besuperimposed on the data of the white background image. The imageprocessing unit 14 outputs data, on which the synthesis processing wasexecuted, as data of a final output image.

FIG. 7 shows an example of an image after setting a margin region 140 tobe added to an ink-wash-painting-like image 120, with theink-wash-painting-like image 120 and a white background image 130. Inthe example shown in FIG. 7, the margin region 140 is set as a regionbetween the periphery of the ink-wash-painting-like image 120 and thewhite background image 130 superimposed with the ink-wash-painting-likeimage 120.

More specifically, FIG. 7A shows the ink-wash-painting-like image 120having coordinates (x, y) composed of a width X and a height Y of theimportance barycenter. In this case, the importance barycenter positionratio calculation unit 71 calculates a position ratio (x/X=y/Y) of thecoordinates (x, y) of the importance barycenter 111, with regard to anaspect ratio (X/Y) of the ink-wash-painting-like image 120 having a sizeof the width X and the height Y. Next, as shown in FIG. 7B, the marginregion setting unit 62 sets a width A and a height B for a size of thewhite background image 130 having an aspect ratio (A/B) that isidentical to the aspect ratio (X/Y) of the ink-wash-painting-like image120. In this case, an equation (X/Y)=α(A/B) is established (where arepresents a margin degree).

The margin degree (α) refers to a degree of a size (an area) of themargin region 140 that is set between the ink-wash-painting-like image120 and the white background image 130. The margin degree can be freelyset as appropriate by the user operating the input unit (not shown). Ina case in which the margin degree is set high based on the operation bythe user, the margin region setting unit 62 sets the size (the area) ofthe margin region 140 large, and in a case in which the margin degree isset low, the margin region setting unit 62 sets the size (the area) ofthe margin region 140 small. The background image position ratiocalculation unit 72 calculates a position ratio (a/A=b/B) of the whitebackground image 130, which would be the same position ratio as theposition ratio (x/X=y/Y) of the importance barycenter 111 calculated bythe importance barycenter position ratio calculation unit 71. In otherwords, (a, b) are calculated such that Equations (5) to (7) as followsare established.

(X/Y)=α(A/B)   (5)

(x/X=y/Y)   (6)

(a/A=b/B)   (7)

(where (a, b) represent coordinates (a, b) of the barycenter 131 of thewhite background image 130).

According to above Equations (5) to (7), when the position ratio of theimportance barycenter 111 of the ink-wash-painting-like image 120 (theoriginal image) is deviated to one side, the barycenter 131 of the whitebackground image 130 will also be deviated to the same side. Moreover,in a case in which the importance is biased toward a certain portion,the background image position ratio calculation unit 72 calculates suchthat the margin region 140 on a side of such a portion is small, and themargin region 140 on a side far from such a center of importance islarge. Therefore, the ink-wash-painting-like image 120 (the originalimage) can be displayed by respecting an object that is desired toreceive attention. In addition, as shown in FIG. 7C, the margin regionsetting unit 62 executes the synthesis processing of synthesizing eachdata of the ink-wash-painting-like image 120 and the white backgroundimage 130, such that the coordinates (x, y) of the importance barycenter111 of the ink-wash-painting-like image 120 (the original image)coincide with the coordinates (a, b) of the barycenter 131 of the whitebackground image 130.

Next, the ink-wash-painting-like image processing executed by the imageprocessing apparatus 1 having such a functional configuration shown inFIG. 2 is described.

FIG. 8 is a flowchart illustrating an example of a flow of theink-wash-painting-like image processing.

When the original image acquisition unit 41 acquires data of an originalimage, the ink-wash-painting-like image processing is initiated, and asequence of processing is executed as follows.

In Step S11, the original image acquisition unit 41 acquires data of theoriginal image.

In Step S12, the characteristic region detection unit 42 executessmoothing processing on the data of the original image. The smoothingprocessing refers to image processing, in which noise and fine textureincluded in the original image are considered to be unnecessary in anink-wash-painting-like image, and are therefore removed or reduced froman original image, thereby smoothing the original image such that theedge representing the contour of the subject is preserved.

In Step S13, the importance information calculation unit 43 executesimportance region calculation processing to be described below withreference to FIG. 9, thereby calculating importance information of theoriginal image from the data of the original image acquired by theoriginal image acquisition unit 41.

In Step S14, the image processing unit 14 executes monochromaticprocessing on the data of the original image, thereby generating data ofthe edge image that is made binary (black and white).

In Step S15, the ink-wash painting conversion unit 44 executesink-wash-painting-like image generation processing to be described belowwith reference to FIG. 10, thereby converting the data of the edge imageinto data of the ink-wash-painting-like image.

In Step S16, the gradation unit 61 executes overall gradation processingto be described below with reference to FIG. 11, thereby calculating anamount of gradation in each pixel, and executing the gradationprocessing on the entire data of the ink-wash-painting-like image, basedon the amount of gradation calculated for each pixel.

In Step S17, the margin region setting unit 62 executes margin regionsetting processing to be described below with reference to FIG. 12,thereby executing synthesis processing of synthesizing each data of theink-wash-painting-like image, on which the gradation processing wasexecuted, and the white background image, based on the importancebarycenter of the ink-wash-painting-like image (the importancebarycenter of the original image) and the barycenter of the whitebackground image.

In Step S18, the image processing unit 14 stores the data of theink-wash-painting-like image with the margin region being set, i.e. thedata of the image, in which the margin region was added to theink-wash-painting-like image, on which the gradation processing wasexecuted, into the storage unit 19 shown in FIG. 1.

It should be noted that the size of the image with the margin regionadded to the ink-wash-painting-like image, i.e. the resolution, may beidentical to the resolution of the original image, and may be differentfrom the resolution of the original image. However, in a case in whichthe size is made identical to the size of the original image, the imageprocessing unit 14 executes reduction processing on the data of theimage with the margin region added to the ink-wash-painting-like image,such that the size coincides with the size of the original image.

As a result, the ink-wash-painting-like image processing is completed.

The flow of the ink-wash-painting-like image processing has beendescribed above with reference to FIG. 8.

Next, regarding the ink-wash-painting-like image processing shown inFIG. 8, a detailed flow of the importance region calculation processingin Step 13 is described with reference to FIG. 9.

FIG. 9 is a flowchart illustrating the importance region calculationprocessing.

In Step S31, the importance calculation unit 51 calculates importance ofeach pixel of the original image. For example, in the example shown inFIG. 3, importance of each pixel 101 of the original image 100 iscalculated.

In Step S32, the importance barycenter calculation unit 53 calculates animportance barycenter of the original image, based on the importance ofeach pixel thus calculated. For example, in the example shown in FIG. 3,an importance barycenter 11 of the original image 100 is calculatedbased on the importance of each pixel 101.

In Step 33, the importance region calculation unit 52 calculates animportance region of the original image, based on pixels havingimportance of at least a threshold value. For example, in the exampleshown in FIG. 3, the importance region 102 of the original image 100 iscalculated from the pixels 101.

As a result, the importance region calculation processing is terminated,i.e. the processing in Step S13 shown in FIG. 8 is terminated, and theprocessing advances to Step S14.

The flow of the importance region calculation processing has beendescribed above with reference to FIG. 9.

Next, regarding the ink-wash-painting-like image processing shown inFIG. 8, a detailed flow of the ink-wash-painting-like image generationprocessing in Step S15 is described with reference to FIG. 10.

FIG. 10 is a flowchart illustrating the ink-wash-painting-like imagegeneration processing.

In Step S51, the characteristic region detection unit 42 searches thedata of the edge image for an edge.

In Step S52, the characteristic region detection unit 42 determineswhether an edge exists, based on a result of the processing in Step S51.

In a case in which an edge exists, in Step S53, the characteristicregion detection unit 42 traces the edge.

More specifically, the characteristic region detection unit 42 scans theedge image in a so-called raster sequence from the upper left, andsearches for pixels belonging to the edge region (Step S51). In a casein which a pixel belonging to such an edge region exists (Step S52:YES), the characteristic region detection unit 42 traces the edge so asto search for other pixels belonging to the edge region (Step S53).

In Step 54, based on the edge region traced in this manner, the ink-washpainting conversion unit 44 determines a brushstroke pattern to be usedin subsequent steps, from among brushstroke patterns, of which data isstored in the brushstroke pattern storage unit (not shown).

More specifically, the ink-wash painting conversion unit 44 determinesan upright-brush brushstroke pattern as the brushstroke pattern to beused for the edge region that was traced in the processing in Step S53.Moreover, the ink-wash painting conversion unit 44 determines anoblique-brush brushstroke pattern as the brushstroke pattern to be usedfor the surrounding region of the edge region. Furthermore, in a case inwhich the edge region is a region of a closed curve, the ink-washpainting conversion unit 44 determines the oblique-brush brushstrokepattern as the brushstroke pattern to be used for the inside of theclosed curve.

In Step S55, the ink-wash painting conversion unit 44 executesprocessing of converting the data of the edge region into data of theupright-brush brushstroke pattern, such that the upright-brushbrushstroke pattern is applied to the edge region that was traced in theprocessing in Step S53 (hereinafter referred to as “applicationprocessing”).

More specifically, since the length of the edge region traced in theprocessing in Step S53 (the length of a longitudinal curve) is differenteach time, the ink-wash painting conversion unit 44 enlarges or reducesthe data of the upright-brush brushstroke pattern that is read from thebrushstroke pattern storage unit (not shown), in accordance with thelength of the edge region. In addition, the ink-wash painting conversionunit 44 converts (applies) the data of the edge region into (to) thedata of the upright-brush brushstroke pattern that was enlarged orreduced.

In Step S56, based on the importance region calculated in the processingin Step S33 (FIG. 9), the ink-wash painting conversion unit 44calculates a drawing color of the upright-brush brushstroke pattern, onwhich the application processing was executed in Step S55. The drawingcolor, i.e. the gray level of the ink, is determined in accordance withthe importance of the original image. Since the importance has a valuefor each pixel, when the drawing color is determined, the ink-washpainting conversion unit 44 calculates an average of the importance in aregion to be drawn with a brush. The ink-wash painting conversion unit44 draws in a darker color (a color closer to black) as the averageimportance thus calculated is higher, and draws in a lighter color (acolor closer to white) as the average importance is lower. Also inprocessing in Steps S59 and S63 to be described below, similarly to theprocessing in Step S56, the ink-wash painting conversion unit 44calculates a drawing color based on an average of the importance.

In Step S57, based on the importance region calculated in the processingin Step S33 (FIG. 9), the ink-wash painting conversion unit 44 executesgradation processing on the upright-brush brushstroke pattern, on whichthe application processing was executed in Step S55. In this case,similarly to the manner in which the ink-wash painting conversion unit44 determines a gray level of the ink based on the importance of theimage, the ink-wash painting conversion unit 44 determines an amount(width) of gradation based on the importance of the image. When theimportance is higher, the color of the ink is darker, and thus theink-wash painting conversion unit 44 reduces the amount of gradation;and when the importance is lower, the color of the ink is lighter, andthus the ink-wash painting conversion unit 44 increases the amount ofgradation. Also in processing in Steps S60 and S64 to be describedbelow, similarly to the processing in Step S57, the ink-wash paintingconversion unit 44 executes gradation processing based on theimportance.

In Step S58, the ink-wash painting conversion unit 44 executesapplication processing, such that the oblique-brush brushstroke patternis applied to the surrounding region of the edge region traced in theprocessing in Step S53.

In Step S59, based on the importance region calculated in the processingin Step S33 (FIG. 9), the ink-wash painting conversion unit 44calculates a drawing color of the oblique-brush brushstroke pattern, onwhich the application processing was executed in Step S58.

In Step S60, based on the importance region calculated in the processingin Step S33 (FIG. 9), the ink-wash painting conversion unit 44 executesgradation processing on the oblique-brush brushstroke pattern, on whichthe application processing was executed in Step S58.

In Step S61, the ink-wash painting conversion unit 44 determines whetherthe edge region traced in the processing in Step S53 is a region of aclosed curve.

In a case in which the edge region is not a region of a closed curve,i.e. in a case in which the region is a region with a starting point andan ending point, the determination in Step S61 is NO, the processingreturns to Step S51 in which another edge is searched, and theprocessing in and after Step S52 is repeated.

On the other hand, in a case in which the edge region is a region of aclosed curve, i.e. in a case in which the region is a region without astarting point and an ending point, the determination in Step S61 isYES, and the processing advances to Step S62.

In Step S62, the ink-wash painting conversion unit 44 executesapplication processing, such that the oblique-brush brushstroke patternis applied to the inside of the closed curve of the edge region tracedin the processing in Step S53.

In Step S63, based on the importance region calculated in the processingin Step S33 (FIG. 9), the ink-wash painting conversion unit 44calculates a drawing color of the oblique-brush brushstroke pattern, onwhich the application processing was executed in Step S62.

In Step S64, based on the importance region calculated in the processingin Step S33 (FIG. 9), the ink-wash painting conversion unit 44 executesgradation processing on the oblique-brush brushstroke pattern, on whichthe application processing was executed in Step S62.

Subsequently, the processing returns to Step S51 in which another edgeis searched, and the processing in and after Step S52 is repeated.

In this way, the loop processing in Steps S51 to S64 is repeatedlyexecuted on each edge region included in the edge image. In addition,when the processing on the last edge region is completed, since an edgecannot be searched in the processing in the next Step S51, thedetermination in the next Step S52 is NO, and the ink-wash-painting-likeimage generation processing is terminated. In other words, theprocessing in Step 15 shown in FIG. 8 is terminated.

The flow of the ink-wash-painting-like image generation processing hasbeen described above with reference to FIG. 10.

Next, regarding the ink-wash-painting-like image processing shown inFIG. 8, a detailed flow of the overall gradation processing in Step S16is described with reference to FIG. 11.

FIG. 11 is a flowchart illustrating the overall gradation processing.

In Step S81, the gradation unit 61 acquires importance barycenterinformation calculated by the importance barycenter calculation unit 53.

In Step S82, the gradation unit 61 acquires information of theimportance region calculated by the importance region calculation unit52.

In Step S83, the gradation unit 61 sets one pixel in the importanceregion of the original image as an attention pixel.

In Step S84, the gradation unit 61 calculates an amount of gradation,based on the importance of the attention pixel, and based on a distancebetween each pixel and the importance barycenter acquired in Step S81.

In Step S85, the gradation unit 61 determines whether all pixels in theimportance region have been set as attention pixels. In a case in whichall pixels in the importance region have not been set as attentionpixels in Step S83, the determination in Step S85 is NO, and theprocessing returns to Step S83. In other words, the processing in StepsS83 to S85 is repeated until the amount of gradation is calculated forall pixels. On the other hand, in a case in which all pixels in theimportance region have been set as attention pixels in Step S83, thedetermination in Step S85 is YES, and the processing advances to StepS86.

In Step S86, the gradation unit 61 executes gradation processing ofgradating the entire image of the original image, based on the gradationamount calculated for each pixel in Step S84. When this processing isterminated, the overall gradation processing is terminated, i.e. theprocessing in Step 16 shown in FIG. 8 is terminated, and the processingadvances to Step S17.

The flow of the overall gradation processing has been described abovewith reference to FIG. 11.

Next, regarding the ink-wash-painting-like image processing shown inFIG. 8, a detailed flow of the margin region setting processing in StepS17 is described with reference to FIG. 12.

FIG. 12 is a flowchart illustrating the margin region settingprocessing.

In Step S101, the importance barycenter position ratio calculation unit71 calculates a position ratio of the importance barycenter of theink-wash-painting-like image. For example, in the example shown in FIG.7, a position ratio of the importance barycenter 111 of the originalimage 100 is calculated.

In Step S102, the margin region setting unit 62 sets a level of themargin degree of the margin region 140 shown in FIG. 7, based on theoperation by the user. For example, in the example shown in FIG. 7, alevel of the margin degree of the margin region 140 is set.

In Step S103, the margin region setting unit 62 sets a size of the whitebackground image. For example, in the example shown in FIG. 7, a size ofthe white background image 130 is set.

In Step S104, the background image position ratio calculation unit 72calculates a barycenter of the white background image. For example, inthe example shown in FIG. 7, the barycenter 131 of the white backgroundimage 130 is calculated.

In Step S105, the synthesis unit 73 executes synthesis processing ofsynthesis and superimposition such that a position of the coordinates ofthe barycenter of the white background image coincides with a positionof the coordinates of the importance barycenter of theink-wash-painting-like image, based on the barycenter of the whitebackground image and the importance barycenter of theink-wash-painting-like image. For example, in the example shown in FIG.7, the synthesis processing of synthesis and superimposition isexecuted, such that the positions of the coordinates of the barycenter131 and the importance barycenter 111 coincide with each other, based onthe barycenter 131 of the white background image 130 and the importancebarycenter 111 of the original image 100. When this processing isterminated, the margin region setting processing is terminated, i.e. theprocessing in Step 17 shown in FIG. 8 is terminated, and the processingadvances to Step S18.

The image processing apparatus of the first embodiment configured asabove includes the original image acquisition unit 41, thecharacteristic region detection unit 42, the importance informationcalculation unit 43, the ink-wash painting conversion unit 44, and theconversion unit 45.

The original image acquisition unit 41 acquires data of the originalimage 100.

The ink-wash painting conversion unit 44 converts the data of theoriginal image 100 acquired by the original image acquisition unit 41into data of the ink-wash-painting-like image 120.

The characteristic region detection unit 42 detects a characteristicregion (an edge region in the present embodiment) of the original image100 from the data of the original image 100 acquired by the originalimage acquisition unit 41.

The importance information calculation unit 43 calculates importanceinformation of the original image 100, based on the data of the originalimage 100 acquired by the original image acquisition unit 41.

As the image processing of further converting the data of theink-wash-painting-like image that was converted by the ink-wash paintingconversion unit 44, the conversion unit 45 executes the gradationprocessing of gradating the characteristic region detected by thecharacteristic region detection unit 42, and the margin settingprocessing of setting the margin region 140 to be added to theink-wash-painting-like image, based on the importance informationcalculated by the importance information calculation unit 43.

In this way, as a result of executing the gradation processing on theink-wash-painting-like image 120 based on the importance information ofthe original image 100, a natural brush drawing of an ink-wash paintingcan be faithfully expressed. As a result, it is possible to obtain animage that is more similar to a real ink-wash painting.

Similarly, as a result of executing the margin setting processing ofsetting the margin region 140 to be added to the ink-wash-painting-likeimage 120 based on the importance information of the original image 100,a natural composition of an ink-wash painting can be faithfullyexpressed. As a result, it is possible to obtain an image that is moresimilar to a real ink-wash painting.

The importance information calculation unit 43 of the image processingapparatus of the present embodiment includes the importance calculationunit 51 and the importance barycenter calculation unit 53.

The importance calculation unit 51 calculates importance of each pixel101 composing the original image 100.

Based on the importance of each pixel 101 calculated by the importancecalculation unit 51, the importance barycenter calculation unit 53calculates the importance barycenter 111 of the original image 100. Inaddition, based on the importance barycenter calculated by theimportance barycenter calculation unit 53, the conversion unit 45executes conversion of the data of the ink-wash-painting-like image 120.

In this way, as a result of executing the conversion of the data of theink-wash-painting-like image 120 based on the importance barycenter, anobject drawn as an ink-wash painting can be expressed based on a drawingmethod peculiar to the ink-wash painting. As a result, it is possible toobtain an image that is further remarkably similar to a real ink-washpainting.

The conversion unit 45 of the image processing apparatus of the presentembodiment includes the gradation unit 61.

The gradation unit 61 executes the gradation processing on thecharacteristic region detected by the characteristic region detectionunit 42, and an object(s) in at least one region in the characteristicregion, based on the importance barycenter calculated by the importancebarycenter calculation unit 53.

In this way, as a result of executing the gradation processing based onthe importance barycenter, an object drawn as an ink-wash painting canbe expressed based on a drawing method peculiar to the ink-washpainting. As a result, it is possible to obtain an image that is furtherremarkably similar to a real ink-wash painting.

The conversion unit 45 of the image processing apparatus of the presentembodiment includes the importance barycenter position ratio calculationunit 71 and the background image position ratio calculation unit 72.

The importance barycenter position ratio calculation unit 71 calculatesa position ratio of the importance barycenter calculated by theimportance barycenter calculation unit 53.

Based on the position ratio of the importance barycenter calculated bythe importance barycenter position ratio calculation unit 71, thebackground image position ratio calculation unit 72 calculates abarycenter position ratio of a white background image 130.

Based on the position ratio of the importance barycenter calculated bythe importance barycenter position ratio calculation unit 71, and basedon the position ratio of the barycenter of the white background image130 calculated by the background image position ratio calculation unit72, the conversion unit 45 synthesizes the data of the white backgroundimage 130 and the data of the ink-wash-painting-like image, therebysetting the margin region 140 to be added to the ink-wash-painting-likeimage 120.

In this way, as a result of setting the margin region 140 of theink-wash-painting-like image 120 based on the position ratio of theimportance barycenter, a natural composition of an ink-wash painting canbe faithfully expressed. As a result, it is possible to obtain an imagethat is more similar to a real ink-wash painting.

The importance information calculation unit 43 of the image processingapparatus of the present embodiment further includes the importanceregion calculation unit 52.

The importance region calculation unit 52 calculates the importanceregion 102, based on the importance calculated for each pixel 101 by theimportance calculation unit 51.

In addition, based on the importance of each pixel 101 in the importanceregion 102 calculated by the importance region calculation unit 52, andbased on a distance between each pixel and the importance barycenter 111calculated by the importance barycenter calculation unit 53, thegradation unit 61 calculates an amount of gradation in each pixel.

By executing the gradation processing in accordance with the importanceof each pixel 101 and the distance from the importance barycenter inthis way, it is possible to appropriately express a blur of an inkpeculiar to an ink-wash painting. As a result, it is possible to obtainan image that is further remarkably similar to a real ink-wash painting.

The image processing apparatus 1 according to the first embodiment ofthe present invention has been described above.

Second Embodiment

Next, an image processing apparatus 1 according to a second embodimentof the present invention is described.

The image processing apparatus 1 according to the second embodiment ofthe present invention can have a hardware configuration and a functionalconfiguration that are basically similar to those of the imageprocessing apparatus 1 according to the first embodiment.

Therefore, FIG. 1 is also a block diagram showing the hardwareconfiguration of the image processing apparatus 1 according to thesecond embodiment.

Furthermore, ink-wash-painting-like image processing, importance regioncalculation processing, and ink-wash-painting-like image generationprocessing executed by the image processing apparatus 1 according to thesecond embodiment are basically similar to the flows of those processingaccording to the first embodiment. Therefore, FIGS. 8, 9 and 10 are alsoflowcharts illustrating the ink-wash-painting-like image processing, theimportance region calculation processing, and the ink-wash-painting-likeimage generation processing according to the second embodiment.

FIG. 13 is a functional block diagram showing a functional configurationfor executing overall gradation processing according to the secondembodiment, regarding the functional configuration of the imageprocessing apparatus 1 of the present invention.

When FIG. 2 is compared with FIG. 13, the functional configuration ofthe image processing unit 14 of the image processing apparatus 1according to the second embodiment is basically similar to that of theimage processing apparatus 1 according to the first embodiment, exceptin a case in which there are a plurality of importance regions, theplurality of importance regions are integrated as an integratedimportance region; therefore, descriptions thereof are omitted. In otherwords, the gradation unit 61 of the image processing apparatus 1 of thefirst embodiment calculates an amount of gradation in each pixel, basedon the importance of each pixel inside the importance region, and adistance between each pixel and the importance barycenter.

On the other hand, in a case in which there are a plurality ofimportance regions, the gradation unit 61 of the image processingapparatus 1 of the second embodiment integrates the plurality ofimportance regions as an integrated importance region, and calculates anamount of gradation inside such an importance region, based onprocessing similar to that in the first embodiment. In addition, outsidethe importance region, the gradation unit 61 of the image processingapparatus 1 of the second embodiment calculates an amount of gradationin each pixel, based on the importance of each pixel inside theintegrated importance region, a distance between each pixel and theimportance barycenter, and the importance barycenter as well as an edgedirection of the integrated importance region.

It should be noted that the units including the original imageacquisition unit 41 to the ink-wash painting conversion unit 44 as wellas the margin region setting unit 62 in the conversion unit 45 of theimage processing apparatus 1 of the second embodiment are similar to theunits including the original image acquisition unit 41 to the ink-washpainting conversion unit 44 as well as the margin region setting unit 62in the conversion unit 45 of the image processing apparatus 1 of thefirst embodiment, respectively; therefore, detailed descriptions thereofare omitted, and only different points are described.

FIG. 13 is a functional block diagram showing a functional configurationfor executing overall gradation processing according to the secondembodiment, regarding the functional configuration of the imageprocessing apparatus shown in FIG. 1.

In a case in which there is a single importance region, the overallgradation processing according to the second embodiment is executedsimilarly to the gradation processing of the first embodiment. Inaddition, in a case in which there are a plurality of importanceregions, the gradation unit 61 integrates the plurality of importanceregions as an integrated importance region, and calculates an amount ofgradation in each pixel, based on the importance of each pixel insidethe integrated importance region, a distance between each pixel and theimportance barycenter, and the importance barycenter as well as an edgedirection of the integrated importance region. In addition, thegradation unit 61 executes gradation processing on the entire image ofthe ink-wash-painting-like image, based on the amount of gradation thuscalculated.

In the second embodiment, the gradation unit 61 further includes animportance region integration unit 81 for executing the overallgradation processing according to the second embodiment.

In a case in which there are a plurality of importance regionscalculated by the importance region calculation unit 52, the importanceregion integration unit 81 integrates the plurality of importanceregions calculated by the importance region calculation unit 52 as anintegrated importance region.

Regarding the integrated importance region, inside the importanceregion, the gradation unit 61 of the second embodiment calculates anamount of gradation in each pixel, based on the importance of each pixelin the importance region, and a distance between each pixel and theimportance barycenter calculated by the importance barycentercalculation unit; and outside the importance region, the gradation unit61 of the second embodiment calculates an amount of gradation in eachpixel, based on the importance of each pixel inside the integratedimportance region, a distance between each pixel and the importancebarycenter calculated by the importance barycenter calculation unit, andthe importance barycenter as well as an edge direction of the integratedimportance region.

FIG. 14 shows an example of data of an original image 100 of the secondembodiment.

In FIG. 14, gray regions are a plurality of importance regions 102A and102B, among each of a plurality of pixels 101 composing the originalimage 100. In addition, an integrated importance region 202 integratedfrom the plurality of importance regions 102A and 102B is shown in ablack frame. Moreover, an importance barycenter 211 calculated based onthe integrated importance region 202 is shown in a substantially centralportion of the integrated importance region 202.

FIG. 15 is a flowchart illustrating the overall gradation processing inStep S16, regarding the ink-wash-painting-like image processing shown inFIG. 8.

In Step S121, the gradation unit 61 acquires importance barycenterinformation calculated by the importance barycenter calculation unit 53.

In Step S122, the gradation unit 61 acquires information of theimportance region calculated by the importance region calculation unit52.

In Step S123, the gradation unit 61 determines whether there areplurality of importance regions calculated by the importance regioncalculation unit 52. In a case in which the importance region isdetermined to be not plural, i.e. to be singular, the processingadvances to Step S130, and processing similar to the overall gradationprocessing in Steps S83 to S85 shown in FIG. 11 is executed. In thiscase, since processing in Steps S130 to S132 of the second embodiment issimilar to the processing in Steps S83 to S85 of the first embodiment,descriptions thereof are omitted. On the other hand, in a case in whichit is determined that there are a plurality of importance regions, theprocessing advances to Step S124.

In Step S124, the importance region integration unit 81 integrates theplurality of importance regions 102A and 102B shown in FIG. 14 as theintegrated importance region 202.

In Step S125, the gradation unit 61 sets one pixel 101 in the integratedimportance region 202 of the original image 100 shown in FIG. 14 as anattention pixel.

In Step S126, the gradation unit 61 determines whether the attentionpixel that was set in Step S125 is a pixel in the importance region. Forexample, in the example shown in FIG. 14, the gradation unit 61determines whether the attention pixel is a pixel 101 in the importanceregion 102A or 102B. In a case in which the attention pixel is a pixelin the importance region, the processing advances to Step S127, andprocessing similar to the overall gradation processing in Step S84 ofthe first embodiment shown in FIG. 11 is executed. In this case, sinceprocessing in Step S127 of the second embodiment is similar to theprocessing in Step S84 of the first embodiment, descriptions thereof areomitted. On the other hand, in a case in which it is determined that theattention pixel is not a pixel in the importance region, the processingadvances to Step S128.

In Step S128, the gradation unit 61 calculates an amount of gradation,based on the importance of the attention pixel, a distance from theimportance barycenter acquired in Step S121, and the importancebarycenter as well as an edge direction of the integrated importanceregion.

In Step S129, the gradation unit 61 determines whether all pixels in theintegrated importance region have been set as attention pixels. In acase in which all pixels in the integrated importance region have notbeen set as attention pixels in Step S125, the determination in StepS129 is NO, and the processing returns to Step S125. In other words, theprocessing in Steps S125 to S129 is repeated until the amount ofgradation is calculated for all pixels in the integrated importanceregion. On the other hand, in a case in which all pixels in theintegrated importance region have been set as attention pixels in StepS125, the determination in Step S129 is YES, and the processing advancesto Step S133.

In Step S133, the gradation unit 61 executes gradation processing ofgradating the entire image of the original image, based on the gradationamount calculated for each pixel in Steps S128, S128 and S131. When thisprocessing is terminated, the overall gradation processing isterminated, i.e. the processing in Step 16 shown in FIG. 8 isterminated, and the processing advances to Step S17.

The gradation unit 61 of the image processing apparatus of the secondembodiment configured as above includes the importance regionintegration unit 81.

In a case in which there are a plurality of importance regionscalculated by the importance region calculation unit 52, the importanceregion integration unit 81 integrates the plurality of importanceregions 102A and 102B shown in FIG. 14 calculated by the importanceregion calculation unit 52 as the integrated importance region 202.

Regarding the integrated importance region 202, inside the importanceregions 102A and 102B, the gradation unit 61 calculates an amount ofgradation in each pixel 101, based on the importance of each pixel 101in the importance regions 102A and 102B, and a distance between eachpixel 101 and the importance barycenter 211 calculated by the importancebarycenter calculation unit 53; and outside the importance regions 102Aand 102B, the gradation unit 61 calculates an amount of gradation ineach pixel 101, based on the importance of each pixel 101 inside theintegrated importance region 202, a distance between each pixel 101 andthe importance barycenter 211 calculated by the importance barycentercalculation unit 53, and the importance barycenter 211 as well as anedge direction of the integrated importance region 202. In the originalimage 100, an important pixel 101 being at a distance from theimportance barycenter may be determined to be not very importantalthough it is actually important. On the other hand, in the presentembodiment, in a case in which there are a plurality of importanceregions 102A and 102B, the gradation unit 61 integrates the plurality ofimportance regions 102A and 102B as the integrated importance region202. In this way, as a result of calculating the amount of gradation inconsideration for the importance of each pixel 101 based on theplurality of importance regions, an object drawn as an ink-wash paintingcan be expressed based on a drawing method peculiar to the ink-washpainting. As a result, it is possible to obtain an image that is furtherremarkably similar to a real ink-wash painting.

It should be noted that the present invention is not limited to theembodiment described above, and any modifications and improvementsthereof within the scope that can realize the object of the presentinvention are included in the present invention.

For example, the importance calculation unit 51 calculates importance,based on color information, brightness (brightness information), orinformation of an edge direction, within a predetermined range aroundeach pixel; however, it is not limited thereto. For example, a techniquecan also be employed, in which the importance calculation unit 51 uses aspatial frequency in a particular region for the attention pixel toincrease importance of a corresponding portion. Moreover, regarding dataof a single original image, the importance calculation unit 51 scans aface search frame in a predetermined direction, identifiescharacteristic portions (face parts) corresponding to eyes, nose, mouthand the like, and determines whether it is a face, based on a positionalrelationship of each face part. In addition, the importance calculationunit 51 can also calculate predetermined importance in accordance withclassification of a characteristic portion thus determined. Furthermore,the importance calculation unit 51 can also identify a particular objectin a scenery photograph or a portrait to calculate predeterminedimportance in accordance with classification of the object thusidentified. Moreover, the importance calculation unit 51 calculatesimportance of each pixel, but it is not limited thereto, and cancalculate importance of each region.

In addition, for example, based on the position ratio of the importancebarycenter calculated by the importance barycenter position ratiocalculation unit 71, and based on the position ratio of the barycenterof the background image calculated by the background image positionratio calculation unit 72, the margin region setting unit 62 of theconversion unit 45 synthesizes the data of the background image and thedata of the ink-wash-painting-like image, thereby setting the marginregion to be added to the ink-wash-painting-like image; however, it isnot limited thereto. For example, the margin region setting unit 62 ofthe conversion unit 45 may further include an image region division unit(not shown) that divides the data of the ink-wash-painting-like imageinto a plurality of regions. Furthermore, among the regions divided bythe image region division unit (not shown), in a case in which a regionincluding a pixel with high importance calculated by the importancecalculation unit 51 is in contact with the periphery of theink-wash-painting-like image, the margin region setting unit 62 may notadd a margin region to the periphery of such a region. In this way, as aresult of not adding a margin region to the periphery of the regionincluding a pixel with high importance, a natural composition of anink-wash painting can be faithfully expressed. As a result, it ispossible to obtain an image that is more similar to a real ink-washpainting.

Moreover, for example, the amount of gradation calculated by thegradation unit 61 can be calculated so as to maximize an amount ofgradation in the edge, by gradually changing a white-based contrast tozero, based on a predetermined function.

The image processing apparatus of the present invention can be appliedto electronic devices in general that can execute the aforementionedimage processing. More specifically, for example, the present inventioncan be applied to a personal computer, a smart phone, a printer, atelevision, a video camera, a portable navigation device, a cell phonedevice, a portable game device, and the like.

The processing sequence described above can be executed by hardware, andcan also be executed by software.

In other words, the hardware configuration shown in FIG. 2 is merely anillustrative example, and the present invention is not particularlylimited thereto. More specifically, the types of functional blocksemployed to realize the aforementioned functions are not particularlylimited to the example in FIG. 2, so long as the image processingapparatus 1 can be provided with the functions enabling theaforementioned processing sequence to be executed as its entirety.

A single functional block may be configured by a single piece ofhardware, a single installation of software, or any combination thereof.

In a case in which the processing sequence is executed by software, aprogram configuring the software is installed from a network or astorage medium into a computer or the like.

The computer may be a computer embedded in dedicated hardware.Alternatively, the computer may be a computer capable of executingvarious functions by installing various programs, e.g., ageneral-purpose personal computer.

The storage medium containing such a program can not only be constitutedby the removable media 31 shown in FIG. 1 distributed separately fromthe device main body for supplying the program to a user, but can alsobe constituted by a storage medium or the like supplied to the user in astate incorporated in the device main body in advance. The removablemedia 31 is composed of a magnetic disk (including a floppy disk), anoptical disk, a magnetic optical disk, or the like, for example. Theoptical disk is composed of a CD-ROM (Compact Disk-Read Only Memory), aDVD (Digital Versatile Disk), or the like, for example. The magneticoptical disk is composed of an MD (Mini-Disk) or the like. The storagemedium supplied to the user in a state incorporated in the device mainbody in advance may include, for example, the ROM 12 shown in FIG. 1, ahard disk included in the storage unit 19 shown in FIG. 1 or the like,in which the program is recorded.

It should be noted that, in the present specification, the stepsdescribing the program recorded in the storage medium include not onlythe processing executed in a time series following this order, but alsoprocessing executed in parallel or individually, which is notnecessarily executed in a time series.

Although some embodiments of the present invention have been describedabove, the embodiments are merely exemplification, and do not limit thetechnical scope of the present invention. Other various embodiments canbe employed for the present invention, and various modifications such asomission and replacement are possible without departing from the spritsof the present invention. Such embodiments and modifications areincluded in the scope of the invention and the summary described in thepresent specification, and are included in the invention recited in theclaims as well as the equivalent scope thereof.

What is claimed is:
 1. An image processing apparatus, comprising: afirst conversion unit that converts data of an original image into dataof a painterly image; a characteristic region detection unit thatdetects a characteristic region of the original image, from the data ofthe original image; and a second conversion unit that executes gradationprocessing of gradating the characteristic region detected by thecharacteristic region detection unit, and margin setting processing ofsetting a margin region to be added to the painterly image, as imageprocessing of further converting the data of the painterly image thatwas converted by the first conversion unit.
 2. The image processingapparatus according to claim 1, wherein the data of the painterly imageis data of an ink-wash-painting-like image.
 3. An image processingapparatus, comprising: an original image acquisition unit that acquiresdata of an original image; a first conversion unit that converts thedata of the original image acquired by the original image acquisitionunit into data of an ink-wash-painting-like image; a characteristicregion detection unit that detects a characteristic region of theoriginal image, from the data of the original image acquired by theoriginal image acquisition unit; an importance information calculationunit that calculates importance information of the original image, basedon the data of the original image acquired by the original imageacquisition unit; and a second conversion unit that executes gradationprocessing of gradating the characteristic region detected by thecharacteristic region detection unit, and margin setting processing ofsetting a margin region to be added to the ink-wash-painting-like image,based on the importance information calculated by the importanceinformation calculation unit, as image processing of further convertingthe data of the ink-wash-painting-like image that was converted by thefirst conversion unit.
 4. The image processing apparatus according toclaim 3, wherein the importance information calculation unit includes:an importance calculation unit that calculates importance of each pixelcomposing the original image; and an importance barycenter calculationunit that calculates an importance barycenter of the original image,based on the importance of the each pixel calculated by the importancecalculation unit, and wherein the second conversion unit executesconversion of the data of the ink-wash-painting-like image, based on theimportance barycenter calculated by the importance barycentercalculation unit.
 5. The image processing apparatus according to claim4, wherein the second conversion unit includes: a gradation unit thatexecutes the gradation processing on the characteristic region detectedby the characteristic region detection unit, and on an object(s) in atleast one region in the characteristic region, based on the importancebarycenter calculated by the importance barycenter calculation unit. 6.The image processing apparatus according to claim 4, wherein the secondconversion unit includes: an importance barycenter position ratiocalculation unit that calculates a position ratio of the importancebarycenter calculated by the importance barycenter calculation unit; anda background image position ratio calculation unit that calculates abarycenter position ratio of a background image, based on the positionratio of the importance barycenter calculated by the importancebarycenter position ratio calculation unit, and wherein the secondconversion unit synthesizes the data of the background image and thedata of the ink-wash-painting-like image, based on the position ratio ofthe importance barycenter calculated by the importance barycenterposition ratio calculation unit, and based on the position ratio of thebarycenter of the background image calculated by the background imageposition ratio calculation unit, thereby setting the margin region to beadded to the ink-wash-painting-like image.
 7. The image processingapparatus according to claim 4, wherein the importance informationcalculation unit further includes: an importance region calculation unitthat calculates an importance region, based on the importance calculatedfor each pixel by the importance calculation unit, and wherein thegradation unit calculates an amount of gradation in each pixel, based onthe importance of each pixel in the importance region calculated by theimportance region calculation unit, and based on a distance between eachpixel and the importance barycenter calculated by the importancebarycenter calculation unit.
 8. The image processing apparatus accordingto claim 7, wherein the gradation unit further includes: an importanceregion integration unit that integrates a plurality of importanceregions calculated by the importance region calculation unit as anintegrated importance region, in a case in which there are a pluralityof importance regions calculated by the importance region calculationunit, wherein, inside the importance regions of the integratedimportance region, the gradation unit calculates an amount of gradationin each pixel, based on importance of each pixel in the importanceregions, and based on a distance between each pixel and the importancebarycenter calculated by the importance barycenter calculation unit, andwherein, outside the importance regions of the integrated importanceregion, the gradation unit calculates an amount of gradation in eachpixel, based on importance of each pixel inside the integratedimportance region, a distance between each pixel and the importancebarycenter calculated by the importance barycenter calculation unit, andthe importance barycenter as well as an edge direction of the integratedimportance region.
 9. The image processing apparatus according to claim6, wherein the second conversion unit further includes: an image regiondivision unit that divides the data of the ink-wash-painting-like imageinto a plurality of regions, and wherein, among the regions divided bythe image region division unit, in a case in which a region including apixel with high importance calculated by the importance calculation unitis in contact with a periphery of the ink-wash-painting-like image, amargin region is not added to the periphery of the region.
 10. An imageprocessing method performed by an image processing apparatus to executeimage processing on an original image, the method comprising: a firstconverting step of converting data of the original image into data of apainterly image; a characteristic region detecting step of detecting acharacteristic region of the original image, from the data of theoriginal image; and a second converting step of executing gradationprocessing of gradating the characteristic region detected in thecharacteristic region detecting step, and margin setting processing ofsetting a margin region to be added to the painterly image, as imageprocessing of further converting the data of the painterly image thatwas converted by the first converting step.
 11. A storage medium havingstored therein a computer readable program for controlling an imageprocessing apparatus that executes image processing on an originalimage, the program causing a computer to implement functions of: a firstconverting unit that converts data of the original image into data of apainterly image; a characteristic region detection unit that detects acharacteristic region of the original image, from the data of theoriginal image; and a second conversion unit that executes gradationprocessing of gradating the characteristic region detected by thecharacteristic region detection unit, and margin setting processing ofsetting a margin region to be added to the painterly image, as imageprocessing of further converting the data of the painterly image thatwas converted by the first conversion unit.